Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Re-implement] Upload with ".tmp" suffix and rename file name after upload #43

Merged
merged 2 commits into from
Mar 14, 2018

Conversation

sakama
Copy link
Contributor

@sakama sakama commented Mar 9, 2018

This is a revise of #37

Some MA tools are polling if remote files with specific file name exists at SFTP server and will import it when file exists. In this case, MA tool doesn't know if Embulk output process is already completed or not.
With this feature, plugin will upload ".tmp" file first, and rename them to ".csv"(or something) after upload finishes.

I changed implementation to set temporary_filename and real_filename to Embulk TaskReport at SftpFileOutput.commit().
After all tasks completed, plugin will get each file name sets and rename remote files.
Of course, temporary file will be left in remote server "/path/to/file.txt.tmp".
However, this is help full for simple mechanical check.

I created another PR #44 since Apache Commons VFS2 is checking parent directory's permission with SSH when rename and it should cause permission failure #40 if command execution over SSH isn't permitted for SFTP account.

@sakama sakama requested a review from muga March 9, 2018 08:45
@sakama sakama merged commit 3d6d2e2 into master Mar 14, 2018
@sakama sakama deleted the re-implement-file-rename-feature branch March 14, 2018 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

1 participant